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ABSTRACT 


In  the  1950s,  ballistic  vulnerability/lethality  (V/L)  emerged  from  the  study  of  terminal 
ballistics  as  a  focus  on  target  end  state.  By  the  early  1960s,  wargames  evolved  in  which  key 
inputs,  generated  by  the  V/L  community,  were  Probabilities  of  Kill  (PK).  Loss/Exchange 
Ratios  (LERs)  became  a  central  measure  of  mission  effectiveness.  V/L  and  wargames  are 
but  two  of  many  fields  developed  largely  independently  in  what  might  be  characterized  as 
a  bottom-up  evolution. 

There  is  now  a  significant  need  to  develop  and  apply  integrated  analyses  to  determine 
required  disciplines  and  tools.  For  example,  the  interest  in  Systems-of-  Systems  requires 
the  application  of  multiple  disciplines.  Meeting  these  needs  is  not  possible  by  simply 
combining  a  collection  of  “bottom-up"  tools.  There  are  serious  realities  that  challenge  the 
analytic  community  horizontally  (factors  at  a  single  level  of  war)  and  vertically  (factors 
connecting  different  levels  war).  These  include: 

•  Analytic  areas  characterized  by  collections  of  metrics  in  which  sharing  or  exclusion 
across  areas  is  not  understood, 

•  State  changes  to  systems  or  components  of  systems  (i.e.,  due  to  invoked  damage  or 
repair)  are  conflated  with  system  performance  and  further  conflated  with  system 
effectiveness,  and 

•  Mission  contexts  are  absent,  making  effectiveness  estimates  ambiguous  at  best. 

As  a  community,  we  require  a  single,  high-level  abstraction  that  is  logically  linked  to 
lower  dimensions.  We  suggest  that  the  doctrinal  planning  and  decision-making  discipline, 
the  Military  Decision-Making  Process  (MDMP),  can  serve  as  just  such  an  overarching 
abstraction.  When  properly  applied,  both  necessary  and  sufficient  metrics  can  be  identified 
and  shared,  and  exclusionary  uses  of  the  same  metric  in  different  subspaces  illuminated. 
For  the  past  decade,  efforts  have  been  made  to  provide  a  formal  structure  for  the  MDMP 
by  defining  mathematical  levels,  operators,  semantic  usage,  proper  linkages,  and  order  of 
data  instantiation.  We  call  this  the  Missions  &  Means  Framework  (MMF)  and  suggest 
that  it  can  serve  as  an  analytic  metaphor  for  the  MDMP.  Finally,  we  assert  that  the  MMF 
can  be  applied  as  the  overarching  framework  needed  to  identify  requirements  with 
significantly  increased  clarity  as  well  as  helping  to  structure  the  application  of  more 
narrowly  focused  analytic  tools  and  techniques  across  the  Department  of  Defense  (DoD). 
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INTRODUCTION 


After  World  War  II,  the  foundation  of  ballistic  vulnerability/  lethality  (V/L)  study 
emerged  as  an  extension  of  terminal  ballistics.  The  focus  of  V/L  study  was  the  platform 
(e.g.,  tank,  aircraft,  human)  end  state  rather  than  simply  warhead-armor  interaction. 
Various  metrics  were  developed  by  the  V/L  community  typically  under  a  label  of 
"Probability  of  Kill,"  or  "PK."  By  the  early  1960s,  wargames  relying  critically  on  key  PK 
inputs  generated  by  the  V/L  community  evolved.  And  Loss/Exchange  Ratios  (LERs),  a 
measure  of  enemy-to-friendly  platforms  destroyed,  became  a  central  measure  of  mission 
effectiveness. 

The  V/L  analyses  and  wargames  are  but  two  of  many  fields  developed  largely 
independently  in  what  might  be  characterized  as  a  bottom-up  evolution.  Over  the 
intervening  decades,  many  disciplines  have  emerged.  In  addition  to  ballistic  V/L, 
examples  include: 

•  Mission  requirements  specification, 

•  Basic  and  applied  research, 

•  Material  research, 

•  Analysis  of  human  dimension, 

•  Cost  estimation, 

•  Investigation  of  effectiveness, 

•  Logistics, 

•  Battlefield  repair, 

•  Operations  research/sy stems  analysis, 

•  Systems-of-Systems  tradeoffs, 

•  Developmental  &  operational  testing, 

• 

Clearly  these  categories  represent  a  fraction  of  the  important  foci  across  the  DoD.  But  what 
we  can  observe  is  that  these  disciplines,  in  the  main,  have  arisen  independent  of  one 
another,  mostly  in  bottom-up  activities.  There  certainly  has  been  no  overarching  structure 
or  framework  to  which  these  areas  had  to  adhere. 

Is  this  really  a  problem?  In  the  next  section,  we  describe  important  challenges  that  confront 
the  DoD  community  because  key  interdisciplinary  metaphors  should  be  strengthened. 


PROBLEM  DEFINITION 

It  is  widely  recognized  that  military  systems  today  are  more  complex  than  ever  before; 
and  at  the  same  time,  military  operations  present  social  and  political  challenges  far  beyond 
characterization  of  kinetic  effects.  As  a  result,  more  than  ever  across  the  DoD  there  is  a 
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significant  need  to  develop  and  apply  integrated  analyses  to  determine  required  disciplines 
and  tools.  Meeting  these  needs  is  not  possible  by  simply  combining  a  collection  of 
"bottom-up"  analyses.  The  issue  here  is  not  a  problem  with  computer  coding  or  software 
interoperability.  There  are  serious  logical  disconnects  and  incompatibilities  in  the 
underpinning  logic  that  challenge  the  analytic  community  horizontally  across  a  single  level 
of  war.  And  the  problem  exits  vertically  as  well  when  connecting  multiple  levels  of  war. 

Some  of  the  more  notable  byproducts  of  these  practices  are: 

•  Acquisition  programs  are  typically  pursued  without  detailed  explanation  of  the  value 
added  and  associated  risk  assessments  in  operational  context,  relative  to  higher  and 
lower-level  missions. 

•  Effectiveness  analyses  (e.g.,  requirements,  wargames,  test,  evaluation)  are  therefore 
not  structured  in  a  way  that  clearly  relates  system  requirements  to  operational  necessity 
using  approved  doctrinal  terms. 

•  Where  mission  context  is  absent,  attempting  effectiveness  estimates  is  uncertain  at 
best. 

•  Analytic  exercises  in  which  (sequentially)  a]  state  changes  to  systems  or  components 
of  systems  (due  to  invoked  damage  or  repair)  are  b]  conflated  with  system  performance 
and  c]  further  conflated  with  system  effectiveness. 

•  An  array  of  analytic  areas  characterized  by  collections  of  metrics  in  which  commonality 
or  exclusion  across  disciplines  is  not  known. 

•  Because  standard  language  is  often  absent,  there  is  imperfect  communication  of  ideas 
and  even  metrics  across  and  even  within  communities  of  practice. 

A  significant  reason  for  these  problems  is  the  absence  of  formal  mission  descriptions. 

The  results  are  often: 

•  Material  and  soldier  performance  metrics  are  evaluated  with  incomplete  knowledge  of 
risk  versus  reward  trade-offs. 

•  Acquisition  activities  proceed  without  standard,  shareable  performance  and 
effectiveness  metrics. 

Specific  analytic  and  test  activities  are  prosecuted  in  isolation  without  the  ability  to 
integrate  them  holistically. 

•  System-of- System  analyses  proceed  in  the  absence  of  requisite  operational  “team”  context 
obtainable  only  from  formal  operational  specification.  But  operational  context  is  critical  to 
assessing  the  proper  “blends”  of  multiple  disciplines. 

Before  confronting  these  issues  in  the  large,  we  examine  a  comparatively  contained,  well- 
known  class  of  ballistic  warhead/target  events  at  a  tactical  level  of  war. 
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A  TACTICAL  WARFARE  EVENT 


To  illustrate  this  problem  set,  we  review  an  actual  analytic  challenge  that  became 
paramount  to  the  Army  in  the  1980s. 

Up  to  this  time,  program  requirements  specified  total  protection  (i.e.,  generally  admitting 
no  armor  penetration)  for  rounds  up  to  a  certain  level  of  lethality.  Testing  generally 
focused  on  rounds  requiring  100%  protection.  Overmatching  attacks  were  simply 
conceded  events  since  penetration  could  not  be  avoided.  And  generally  for  overmatching 
munitions,  no  tests  were  performed,  particularly  on  full-up  platforms,  particularly  ones 
fully  loaded  with  ammunition  and  fuel.  In  the  early  1980s,  as  a  number  of  major 
Army  programs  was  heading  into  Low-Rate  Initial  Production  (LRIP),  multiple  voices 
raised  valid  concerns  having  to  do  with  the  battlefield  survivability  vis-  a-vis  overmatches. 
Clearly,  armor  overmatch  can  lead  to  a  full  range  of  outcomes  from  trivial  internal 
damage  to  catastrophic  reaction!  It  became  appreciated  that  delineating  the  possibility 
and  likelihood  of  particular  outcomes  could  provide  important  links  to  the  degradation 
or  total  loss  of  system  performance,  including  most  importantly  the  causes  of  loss  or  injury 
to  crew. 

Beginning  with  the  establishment  of  the  Joint  Live-Fire  Test  Charter  (Linder,  1984)  and 
later  reinforced  with  the  National  Defense  Authorization  Act  for  FY  1987  (Live  Fire  Testing, 
1986),  "realistic"  vulnerability  testing  utterly  changed  the  Test  and  Evaluation  (T&E) 
landscape,  bringing  extreme  new  challenges  in  test  methodology,  V/L  modeling  methods, 
and  cost  containment. 

A  key  component  of  the  live-fire  regulations  was  that  computer  simulations  were  to  be  run 
prior  to  test  execution  to  compare  the  test  outcome  with  model  predictions.  The  first 
program  to  follow  this  practice  was  that  of  the  Bradley  Fighting  Vehicle.  The  most 
applicable  vulnerability  model  at  that  time  was  used  to  support  the  program  (Bradley 
Survivability  Enhancement  Program,  1985).  There  were  many  critics  of  the  program,  but 
the  issue  of  relevance  here  is  the  use  of  the  extant  V/L  models  to  provide  shot  predictions 
for  the  actual  field  tests.  The  V/L  model  support  for  the  Bradley  program  was  heavily 
criticized  as  being  unreliable  (General  Accounting  Office,  1987).  The  Abrams  Live-Fire 
Program  followed  quickly,  with  the  same  requirement  to  provide  pre-shot  predictions  for 
more  than  50  munition/target  combinations.  At  the  outset  of  the  modeling  preparations 
for  the  Abrams  program,  workers  examined  the  problems  involved  in  simulating  actual 
live-fire  shots,  and  then  developed  some  fairly  radical  strategies  for  live-fire  prediction. 
Readers  interested  in  the  details  can  peruse  Deitz  and  Ozolins  (1989)  and  Deitz  and  Starks 
(1997). 
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For  our  purposes  now,  however,  the  problems  encountered  in  the  1980s  with  V/L  analysis 
are  illustrated  metaphorically  by  Figure  1.  Here  a  threat/target  interaction  is 
characterized  by  essentially  a  single  black-box  model  in  which  the  round/hit  point  is  input 
to  the  model  and  a  single  PK  comes  out  of  the  computer.  In  this  approach  a  lumped- 
parameter  model  is  invoked  in  which  internal  details  and  intermediate  results  are  hidden 
or  unknown.  A  key  attendant  issue  then,  and  even  to  this  day,  is  the  manner  of  both 
the  generation  and  interpretation  of  the  PKs  in  the  simulation.  Again,  the  reader  is 
directed  to  Deitz  and  Starks  (1997)  for  a  discussion  of  these  issues;  however,  we  simply 
observe  here  that  it  is  highly  unusual  for  the  outcome  of  an  event  (outside  the  field  of  V/L) 
to  be  characterized  by  a  probability  after  the  event  is  played  out! 


( OUTCOME ) 


* 


Ballistic  Live-Fire 
Test/Prediction 


Figure  1.  A  simple,  lumped-parameter  model  abstraction.  An  early  vulnerability 
model  can  be  thought  of  as  having  an  EVENT  initiation  with  an  OUTCOME,  but  no 
intermediate  results. 

During  the  run-up  to  the  Abrams  Live  Fire  tests.  Army  modelers  attempted  to  refine 
the  "black  box"  illustrated  in  Figure  1  by  subdividing  it  into  a  sequence  of  three  distinct 
mathematical  spaces  or  levels,  (forward  time)  connected  by  operators  (shown  as  red 
arrows.  A  new  V/L  model  called  SQuASH  was  developed,  and  it  was  designed  to 
calculate  first  damage,  then  map  the  damage  to  platform  capability,  and  finally  map 
capability  to  Effectiveness  (or  Utility).  The  basic  outline  is  shown  in  Figure  2.  The 
original  structure  attempting  to  formalize  this  level  of  thinking  was  called  the  V/L 
Taxonomy  (Deitz,  1996).  (Note:  The  numbering  schema  for  the  Levels  1-7  is  discussed  in 
Deitz  et  al.,  2009,  as  well  as  later  in  this  text  with  Figure  5,  etc.) 
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Figure  2.  A  metaphor  used  for  the  Abrams  Live  Fire  modeling  effort.  An  EVENT 
results  in  component  damage  at  Level  2.  Damage  at  Level  2  changes  platform 
capability  at  Level  3.  Reduced  capability  leads  to  a  lowering  of  Effectiveness  at  Level 
4,  the  OUTCOME  of  the  complete  process. 

The  decomposition  of  the  V/L  process  leading  to  the  structure  of  the  V/L  Taxonomy  set  the 
stage  for  clarifying  the  associated  test/model  metrics  and  increasing  the  likelihood  of 
model  validation.  However,  a  number  of  key  hindrances  remained: 

•  As  noted  elsewhere,  the  PK  results  in  the  V/L  models  came  from  a  concatenation  of 
Level  3  and  Level  4  metrics  such  that,  due  to  their  intrinsic  subjectivity,  the  results 
could  not  actually  be  compared  with  test  results  from  the  field. 

•  The  inability  to  generate  Level  4  metrics  other  than  PKs  hindered  the  application  of 
V/L  model  results  beyond  wargames  to  a  wider  set  of  analytic  problems. 

•  Other  than  in  a  few  special  studies  (e.g.,  Abell  et  al.,  1990),  neither  in  the  standard 
practice  of  V/L  modeling  nor  in  field  testing  have  significant  efforts  been  directed  to 
the  estimated  or  measured  mappings  between  component  state  space  and  platform 
performance. 

Nevertheless,  the  pursuit  of  this  more  detailed  structure  made  possible  later  refinements 
to  an  advanced  version  of  the  MUVES  SQuASH  model  (Baker  et  al.,  1998),  which  greatly 
enhanced  the  ability  to  compare  intermediate  and  final  model  outputs  to  the  original 
Bradley  live-fire  field  test  results.  A  few  observations: 

•  There  are  many  interaction  mechanisms  beyond  ballistic.  With  the  three-level 
decomposition  shown  in  Figure  2,  it  is  possible  to  develop  component  state  change 
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algorithms  reflecting  other  damage  classes  (e.g.,  high-power  microwave  and  laser 
damage,  and  physics  of  failure);  and  on  the  positive  side,  there  can  be  algorithms 
describing  battle-damage  repair,  resupply,  or  sleep  for  fatigued  warfighters. 

•  The  health  of  the  components  at  Level  2  doesn't  depend  on  the  origin  of  their  state 
change.  Likewise,  once  an  adequate  utility  is  represented  at  Level  4,  the  effect  of 
many  different  phenomenologies  can  be  assessed  by  the  two  latter  mappings. 

•  Finally,  this  structure  of  Figure  2  shows  how  to  integrate  combinations  of  many 
component  change  mechanisms.  As  they  arise  over  a  mission,  the  running  states  of 
component  health  can  be  remapped  to  platform  capability  and  then  to  mission 
effectiveness. 

Even  with  this  comparatively  simple  example  at  a  single  level  of  war,  we  can  see  that 
the  decomposition  of  a  complex  event  into  its  constituent  pieces  can  add  clarity  and  useful 
insight  into  bringing  model  intermediate  and  final  results  into  consonance  with  field- 
testable  metrics.  Nevertheless,  the  structure  illustrated  in  Figure  2  falls  far  short  of  a 
framework  capable  of  dealing  with  the  broader  issues  of  warfighting.  To  conceptualize  a 
more  general  framework,  we  turn  to  the  world  of  the  professional  warfighter. 

A  final  point.  As  a  matter  of  routine,  when  high-resolution  V/L  models  (i.e.,  those  that 
characterize  internal  platform  components  in  detail)  are  exercised,  they  are  typically 
capable  of  providing  intermediate  levels  of  capability  and  (therefore)  intermediate 
effectiveness  too.  However,  typical  practice  is  to  post-process  such  metrics  into  Bernoulli 
PKs.  By  this  means  of  reporting,  any  kind  of  ballistic  interaction  with  a  platform  can 
only  result  in  an  all  or  nothing  outcome.  We  suggest  that  these  (binary)  outcome  metrics 
cannot  provide  the  resolution  required  in  contemporary  test  and  analysis  venues.  More 
on  this  issue  later. 

HOW  ARE  MISSIONS  PROSECUTED? 

We  now  turn  to  the  professional  warfighter  (or  operator)  to  see  how  they  develop, 
prosecute,  and  assess  virtually  all  missions  across  the  Range  of  Military  Operations  (see 
Joint  Publication,  2011).  For  many  years,  warfighters  have  used  the  Military  Decision- 
Making  Process  (MDMP)  (see,  for  example,  Marr,  2001)  as  the  underlying  framework 
for  planning,  structuring,  organizing,  and  executing  all  manner  of  missions,  whether 
"kinetic"  or  not.  Figure  3  illustrates  the  MDMP  developed  to  analyze  a  Military  Operation 
in  Urban  Terrain  (MOUT)  mission  (Harris  et  al.,  2000).  It  consists  of  sequences  of  tasks 
hierarchically  structured  by  level  of  war,  ultimately  from  the  National  Command  Level 
down  to  the  Tactical  Atomic.  The  laydown  of  tasks  can  be  compared  to  the  construct  of 
PERT  or  Gantt  charts  typically  used  in  a  variety  of  applications  for  prosecuting  complex 
projects. 
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Figure  3.  Part  of  a  MOUT  mission  layout  by  level  of  war.  In  general,  the 
higher-level  tasks  are  defined  and  portions  passed  to  lower  levels  for  either 
execution  and/or  further  decomposition.  The  dashed  blue  line  represents  the 
top-down  inferred  relationships.  The  red  arrow  indicates  time  forward. 

Further: 

•  The  MDMP  is  all  about  mission  planning  and  task  execution,  monitoring  results  and 
assessment  of  progress  against  mission  objectives.  Tasks  are  ubiquitous! 

•  A  key  issue  of  semantics  is  solved  since  the  establishment  of  a  Universal  Joint  Task  List 
(UJTL)  (Chairman  of  the  Joint  Chiefs  of  Staff,  2012)  to  specify  and  describe  tasks  (with 
Conditions  and  Standards)  from  the  National  Command  level  down  through  the 
Operational  level  of  exercise.  Each  Service  has  its  own  lists  that  describe  its  Tasks  (with 
Conditions  and  Standards)  from  the  Operational  to  the  Tactical  Atomic  (lowest  fighting 
levels).  The  tasks  for  the  Army  are  defined  by  the  Army  Universal  Task  List 
(Headquarters,  Department  of  the  Army,  2012). 

•  When  informed  by  key  reference  missions  (including  Joint  and  Service  concepts),  the 
MDMP  should  serve  as  the  single  integrating  framework  for  the  Defense  community. 
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•  Materiel  Requirements  should  derive  from  analysis  of  requirements  for  successful 
task  execution,  under  appropriate  conditions  and  standards. 

•  Since  tasks  at  higher  levels  of  war  inform  lower  levels,  there  is  a  top-to-bottom 
traceability,  somewhat  akin  to  a  mathematical  mapping  which  (inferentially)  projects 
higher-level  information  to  lower  levels. 

•  This  inferred  higher-to-lower  mapping  can  serve  to  identify  which  metrics  at  the 
same  level  of  war  are  derived  from  (and  therefore  share  a  common  heritage  with) 
metrics  at  levels  above.  Through  these  mapping  relationships,  when  properly  applied, 
both  necessary  and  sufficient  metrics  can  be  identified  and  shared,  and  exclusionary 
uses  of  the  same  metric  in  different  subspaces  illuminated.  (See  Appendix  A  for  a 
discussion  of  mappings.) 

We  suggest  that  doctrinal  planning  and  decision-making  processes  such  as  the  MDMP, 
when  used  to  define  and  instantiate  a  mission,  can  serve  as  just  such  a  highest-  level 
defining  state  space.  When  properly  applied,  both  necessary  and  sufficient  metrics  can  be 
identified  and  shared  and  exclusionary  uses  of  the  same  metric  in  different  subspaces 
illuminated.  The  Joint  Capabilities  Integration  Development  System  (JCIDS)  is  at  least 
implicitly  based  on  this  premise.  The  Requirement  Identification  and  Document 
Generation  phase  of  the  JCIDS  process  shown  in  Figure  4  directs  Services,  Combatant 
Commands,  and  other  DoD  Components  to  "conduct  Capabilities  Based  Assessments 
(CBAs)  or  other  studies  to  assess  capability  requirements  and  associated  capability  gaps 
and  risks  .  .  .  the  assessments  are  informed  by  high  level  strategy  and  guidance  in 

the  National  Security  Strategy,  National  Defense  Strategy,  National  Military  Strategy  .  .  ." 
It  continues  by  stating  that  "...  capability  requirements  and  capability  gaps  identified 
through  CBAs  and  other  studies  are  traceable  to  an  organization's  assigned  roles  and 
missions,  and,  to  the  greatest  extent  possible,  described  in  terms  of  tasks,  standards,  and 
conditions  ..."  (Chairman  of  the  Joint  Chiefs  of  Staff,  2012). 

THE  MDMP  AS  A  FORMAL  STRUCTURE 


The  first  issue  that  might  arise  is  why  does  the  Military  Decision-Making  Process  need 
a  formal  structure?  The  answer  lies  in  the  MDMP's  target  audience— military  planners 
and  operators.  It  is  a  process  designed  to  guide  mission  analysis,  planning,  and 
assessment  of  ongoing  operations,  and  it  assumes  an  advanced  level  of  familiarity  with 
military  doctrine  and  doctrinal  terms  and  graphics.  The  MDMP  focuses  on  development 
and  production  of  outputs  (briefing  charts,  plans  and  orders,  etc.)  that  are  meant  to  convey 
the  results  of  the  process  to  military  decision-makers,  staffs,  and  executing  organizations. 
These  outputs  are  normally  generated  in  the  form  of  flat  file  text  and  graphics  using  the 
doctrinal  terms  and  graphics  familiar  to  the  target  audience.  For  the  vast  majority  of  the 
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Figure  4.  The  Joint  Capabilities  Integration  Development  System  (JCIDS)  process. 

civilian  and  contractor  workforce  supporting  the  operating  forces  can  be  difficult  to 
understand  and  even  more  difficult  to  apply.  Because  these  products  are  outputs  of  the 
MDMP  process,  there  is  usually  little-  to-no  documentation  of  the  detailed  thinking, 
discussions,  and  subprocesses  that  lead  to  the  final  output.  And  when  such 
documentation  does  exist,  it  is  conveyed  using  the  same  community-specific  language 
not  commonly  understood  in  the  larger  DoD  community.  With  so  much  at  stake,  we 
suggest: 

•  Require  a  Defense-wide  framework,  language,  and  processes  common  to  and  shared 
by  all  participants. 

•  Establish  the  pieces  and  how  they  fit  together. 

•  Resolve  and  extend  semantics  and  syntax  issues;  task  lists  represent  only  a  part  of 
requisite  shared  language. 

•  Identify  objective  elements;  facts,  are  inherently  quantifiable. 

•  Identify  subjective  elements;  expert  opinion,  particularly  as  related  to  mission 
effectiveness,  must  nevertheless  be  framed  using  quantitative  discipline. 

•  Start  with  the  mission,  since  it's  about  mission  success. 

•  Ensure  that  missions  underpinning  the  DoD  enterprise  support  functions  are 
contained  in  high-level  strategy  and  guidance,  joint  and  service  concepts,  and  urgent 
operational  needs  (UONs)  from  combatant  commanders. 

•  Begin  requirements  identification  with  the  application  a  common  framework  to  concept 

analysis  and  the  initial  identification  of  capability  gaps  and  risks. 
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•  Use  a  common  framework  to  enable  effective  integration  across  enterprise  stove 
pipes  through  early  collaboration,  in  depth  understanding  of  the  logic  and  context 
behind  requirements,  greater  vertical  and  horizontal  transparency,  and  a  logical 
structure  for  storing,  organizing,  accessing,  and  updating  requirements  data  across 
system  life  cycles. 

The  MDMP  structures  multiple  levels  of  war  and  does  so  by  linking  tasks  both  horizontally 
(at  a  given  level  of  war)  and  vertically  (by  level  of  war).  The  V/L  Taxonomy  was  originally 
focused  on  a  specific  kind  of  task  execution  and  embodied  the  notion  of  physical  state  space 
(i.e.,  the  platform  or  person),  the  related  capability  of  the  platform,  and  the  "utility"  of  the 
platform.  Upon  the  emergence  of  the  semantics  established  by  the  UJTL  and  Service  Task 
Lists,  it  seemed  that  the  illusive  issue  of  effectiveness  should  turn  on  the  ability  to  execute 
tasks.  Finally,  it  was  clear  that  the  proper  occupant  of  Level  4  (per  Figure  2)  should  be 
tasks,  not  probabilities. 

THE  MISSIONS  &  MEANS  FRAMEWORK  (MMF1 

As  noted  earlier,  the  V/L  Taxonomy,  although  providing  useful  insights  to  single  platform 
encounters,  fell  far  short  of  a  complete  description  of  warfare.  To  remedy  these 
limitations,  the  MMF  (Sheehan  et  al.,  2003)  evolved  from  an  integration  of  the  V/L 
Taxonomy,  the  Task  Lists  construct,  and  additional  requisite  context  information  necessary 
to  define,  execute,  and  monitor  warfare.  These  efforts  resulted  effectively  in  a  structure 
that  can  serve  as  an  analytic  surrogate  for  MDMP.  Further  description  can  be  found  in 
Appendix  E  of  the  monograph  by  Deitz  et  al.  (2009). 

MMF  has  also  been  discussed  elsewhere  (e.g.,  Sheehan  et  al.,  2003;  Deitz  et  al.,  2009, 
Appendix  E;  Ward  et  al.,  2012),  so  the  description  here  will  be  brief.  Depicted  in  Figure  5, 
the  MMF  is  constituted  by  eleven  fundamental  elements;  seven  levels  and  four  operators. 
The  top  three  levels  (Levels  5-7)  are  used  to  describe  the  Mission  context  in  terms  of  what 
is  to  be  accomplished  and  why  (Level  7);  under  what  environmental  conditions  (Level  6); 
and  when  and  where  (Level  5).  The  data  to  be  stored  and  organized  using  these  three 
levels  include  the  results  of  front-end  analysis  processes  for  missions  received  or  derived 
(e.g.,  mission  analysis  and  intelligence  preparation  of  the  mission  space).  This  information 
is  normally  published/found  in  paragraphs  1  (Situation)  and  2  (Mission)  of  the  Operations 
Plan  (OPLAN)  and  in  supporting  annexes.  Once  a  mission  has  been  received/derived  and 
analyzed,  the  MDMP  is  applied  to  guide  the  process  of  developing  a  plan  of  action  to 
accomplish  the  mission  using  the  means  available.  The  remaining  four  levels  (Levels  1-4) 
and  the  four  operators  are  used  to  describe  the  means  as  illustrated  in  Figure  5. 
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6.  Context,  Environment  (Military,  Civil,  Physical,  etc.) 


Planning 


cz£>  Employment 


Figure  5.  The  Missions  &  Means  Framework.  The  MMF  is  characterized  by 
eleven  fundamental  elements:  seven  levels  and  four  operators. 

Note  that  Levels  5  and  6  represent  a  portion  of  the  mission  context  that  is  shared  by  all 
entities  as  represented  by  the  OWNFOR  (Own  Forces)  and  OPFOR  (Opposition  Forces) 
boxes  in  Level  5  even  though  each  entity  may  have  its  own  unique  Level  7  mission  and 
plan  of  action  (Levels  1-4  with  operators).  The  appearance  of  Level  7  above  Level  4  for 
both  sides  represents  the  normal  practice  of  "restating"  the  mission  externally  imposed  by 
a  higher  authority  into  a  mission  statement  for  the  executing  entity.  This  is  also  a  key 
aspect  of  establishing  the  vertical  linkage  between  military  echelons  (e.g.,  company, 
battalion,  brigade,  etc.)  and  levels  of  war  (i.e.,  tactical,  operational,  strategic).  Note  also 
that  Level  1,  in  the  center  of  the  MMF  is  a  shared  space,  representing  the  interactions 
and  resulting  effects  that  task  execution  by  any  side  can  have  on  itself,  other  sides,  and 
environmental  variables.  The  red  arrows  represent  time-forward  operators  which  link 
the  Levels  1  through  4  in  a  time-forward  sequence  as  operations  are  executed  in  live, 
virtual,  or  constructive  fashion.  The  dotted  blue  arrows  represent  the  time-backward 
(or  top-down)  planning  actions  performed  by  mission  planners  during  course  of  action 
development,  wargaming,  and  mission  rehearsal. 
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TASK  PROSECUTION 


We  now  examine  the  explicit  task-processing  processes  that  are  represented  by  the  front- 
most  layer  of  the  Figure  5.  For  simplicity.  Figure  6  shows  the  front  layer  of  task  execution 
entities  for  the  OWNFOR.  Level  4,  shown  in  green,  represents  a  particular  task.  Moving 
clockwise  (time  forward),  the  Oci  Operator  links  to  a  particular  class  of  interaction 
represented  by  Level  1.  The  examples  discussed  earlier  were  ballistic,  but  can  actually 
be  based  on  a  wide  variety  of  phenomenologies.  The  Oci  Operator  when  called 
repeatedly  acts  as  a  time-ordered  event  list,  commonly  used  in  simulations  to  organize 
a  sequence  of  events.  Based  on  the  class  of  interaction  called  at  Level  1,  the  Oi,2  Operator 
causes  changes  to  the  people/materiel  represented  at  Level  2.  Interactions  can  either  be 
"negative"  (causing  damage)  or  "positive"  (fixing  damage). 


|  Tasks  | 


Figure  6.  An  illustration  of  a  Task  Cycle.  A  task  at  Level  4  initiates  an  interaction 
at  Level  1.  The  Oi,2  Operator  changes  the  state  of  the  components  at  Level  2.  A 
new  capability  is  computed  at  Level  3  and  then  finally  compared  with  the 
capability  required  for  the  next  task  in  the  cycle  at  Level  4.  If  the  current 
capability  at  Level  3  meets  or  exceeds  that  called  for  by  the  next  task,  the 
process  continues.  One  Task  Cycle  (i.e.,  one  360°  revolution)  from  initiation  to 
final  capability/task  comparison  via  the  Om  Operator  might  represent  a  single 
Developmental  Test. 
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The  02,3  Operator  takes  the  current  state  of  Level  2  and  maps  it  to  its  current  capability. 
The  geometry  representing  complete  platforms  (both  external  armor  and  interior 
components)  has  been  characterized  by  BRL-CAD®  for  many  years  (see  Anderson  and 
Edwards,  2004,  and  BRL-CAD,  2013).  Key  to  the  analysis  of  V/L  effects  is  a  full 
representation  of  component  geometry  to  include  just  which  warfare  functions  by  platform 
system.  The  characterization  of  key  functions  (e.g.,  firepower,  mobility,  communication, 
etc.)  are  often  represented  by  fault  trees.  Such  trees  underwrite  the  extent  to  which 
components  are  critically  vulnerable  or  have  redundant  support.  When  these  trees  are 
properly  structured,  the  damage  state  of  the  component  systems  can  be  used  to  estimate 
the  level  of  specific  performance  in  a  continuum  from  fully  functional  to  nonfunctional. 

After  a  new  capability  at  Level  3  is  estimated,  it  is  then  compared  to  the  Level  4  by  the  Om 
Operator,  where  a  comparison  is  made  between  the  capability  called  for  in  the  task  (i.e., 
one  element  of  the  mission  requirement),  and  the  current  capability  of  the  materiel 
represented  at  Level  2.  One  complete  revolution  can  be  termed  a  Task  Cycle.  Multiple 
Task  Cycles  can  be  sequentially  and  simultaneously  linked  to  execute  a  set  of  tasks  in  a 
mission  thread.  Sequential  Task  Cycle  linkages  can  be  created  based  on  a  logical  timeline 
generated  during  the  planning  process.  Military  planners  apply  planning  factors  (e.g., 
average  convoy  speed  on  paved  roads)  to  estimate  time  required  for  task  execution  under 
varying  condition  sets.  Wargaming  and  mission  rehearsals  are  also  conducted  to  identify 
and  establish  conditions-based  linkages  (e.g.,  refueling  must  occur  before  convoy 
continues  movement).  Simultaneous  linkages  normally  occur  when  there  are 
dependencies  between  tasks.  Depending  on  the  operational  conditions,  for  example, 
planners  may  determine  that  convoys  only  occur  simultaneously  with  surveillance  and 
reconnaissance  of  the  route  to  minimize  the  risk  of  ambush  or  improvised  explosive  device 
(IED)  attack  during  movement. 

We  make  some  observations  concerning  the  levels  of  the  Task  Cycle.  As  illustrated  in 
Figure  7,  we  note  that  the  components  and  capabilities  are  predominantly  objective. 
Material  elements  are  inherently  physical  and  measureable;  in  application  to  cognitive 
issues,  they  may  be  primarily  subjective.  Tasks  are  inherently  subjective;  they  are 
conceptualized  by  the  mission  planner  and  are  fundamentally  a  matter  of  judgment. 
Interactions,  when  called  in  the  time-forward  (red  arrow)  execution  mode  under  test, 
are  inherently  objective  in  that  they  can  be  observed  and  their  combined  effects  can  be 
measured.  They  obey  the  laws  of  physics,  biology,  and,  to  a  lesser  degree  of  certainty, 
psychology,  and  may  therefore  be  typically  objective.  Interactions  identified  in  the 
planning  mode  may  also  be  subjective  in  that  they  arise  from  subject  matter  expert 
prediction  or  model/simulation  results  of  task  execution.  This  top-down  process  is 


16 


MOSTLY 

SUBJECTIVE 


4.  Tasks,  Operations 


3.  Functions, 
Services 


Capabilities 


Interactions 


2.  Personnel,  Units 
Components,  Systems 


Components 


MOSTLY 

OBJECTIVE 


Figure  7.  Objective/Subjective  partitioning  of  the  MMF.  Within  each  force,  the 
four  front  levels  of  MMF  are  categorized  as  totheir  objective  vice  subjective 
nature.  Material  components/  capabilities  are  inherently  objective;  cognitive 
components,  not!  Tasks  are  inherently  subjective  while  Interactions  are  a  split 
depending  on  whether  they  are  an  operator  (subjective,  top-down)  choice  or  the 
result  of  a  physical  stimulus  (under  time-  forward  exercise). 

represented  by  the  dotted  blue  links  shown  in  Figure  5.  That  is  why  we  show  Level  1, 
Interactions,  spanning  both  categories. 

INTERACTIONS  BETWEEN  OPPOSING  FORCES 

The  MMF  diagram  of  Figure  5  shows  two  opposing  forces.  We  want  to  illustrate  the  four 
basic  variants  of  task  execution.  Figure  8  shows  the  two  forces  with  the  time-  forward 
operators.  Level  1,  Interactions,  is  shown  outside  of  the  force  descriptors  since  they  are 
"owned"  by  no  one,  but  defined  by  the  laws  of  nature. 

When  tasks  are  initiated  by  a  force,  they  can  be  of  two  forms.  Starting  from  the  left  in 
Figure  9,  the  OWN  FOR  can  be  self-directed,  initiate  a  task  on  itself,  operating  a  vehicle 
is  such  an  example  as  it  causes  depletion  of  fuel.  The  second  type  is  an  OWNFOR  task  and 
may  be  outward  directed,  initiated  against  the  OPFOR.  An  example  here  is  a  blue  vehicle 
firing  upon  a  red  target.  The  task  initiated  by  the  OWNFOR  changes  the  state  of  OPFOR 
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Figure  8.  The  operators  of  the  MMF.  The  opposing  forces  of  MMF  are  shown  with  the 
time-forward  operators.  Note  the  OWNFOR  (time-forward)  operators  move 
clockwise;  the  OPFOR  move  counterclockwise. 


OWNFOR  on 

OWNFOR  on 

OPFOR  on 

OPFOR  on 
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OWNFOR 

OPFOR 

Figure  9.  Two  kinds  of  tasks,  the  MMF  self-directed  (the  two  outer  diagrams) 
and  the  two  outward-directed  (the  two  inner  diagrams). 

materiel.  The  third  and  fourth  elements  show  the  two  cases  for  the  OPFOR.  In  point  of 
fact,  tasks  can  be  initiated  on  one's  own  force  without  causing  an  interaction  with  the 
enemy.  However,  it's  hard  to  imagine  how  one  side  can  initiate  a  task  against  the 
opposition  without  having  some  effects  on  itself.  A  shot  from  one  side  to  the  other  may 
cause  target  destruction,  but  it  also  depletes  the  ammo  stores  for  the  side  initiating  the 
interaction.  Outward-directed  interactions  may  also  generate  effects  on  other  entities  and 
on  elements  of  the  commonly  shared  Level  6,  operational  environment.  These  effects 
may  be  intentional  or  unintentional.  For  the  purposes  of  this  paper,  we  are  focusing  on 
effects  that  result  in  state  changes  to  OWNFOR  and/or  OPFOR  Level  2  materiel/people. 
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IMPORTANT  TAKEAWAYS 


It  is  important  to  note  that  the  four  levels  represented  in  the  MMF,  Figure  6,  are  formed 
by  classes  of  metrics  that  are  the  largest  that  they  can  be  while  still  remaining 
homogeneous  as  a  class.  From  a  natural  language  perspective.  Level  4,  tasks,  can  be 
thought  of  as  verbs,  while  Level  2,  materiel,  can  be  thought  of  as  nouns.  Composing  a 
mission,  like  forming  a  sentence,  involves  taking  nouns  and  linking  them  to  verbs.  But  in 
the  MMF  structure,  the  nouns  and  verbs  are  not  connected  directly.  They  are  linked  by 
interactions  on  one  side  and  capabilities  on  the  other.  This  retains  important  flexibility  in 
the  paradigm  so  that  platforms,  for  example,  can  be  modified  according  to  the  interactions 
they  have  experienced.  To  take  the  natural  language  view  further,  the  interactions  may 
be  analogous  to  adjectives  since  they  modify  nouns.  And  functions  (or  capabilities)  might 
be  thought  of  as  adverbs  as  they  modify  verbs. 

In  Figure  1  we  illustrated  a  V/L  estimation  as  a  lumped-parameter  process.  The  standard 
outcome  used  widely  in  the  community  is  a  PK.  As  stated  previously,  the  interpretation 
of  PKs  is  ambiguous  at  best  and  certainly  not  relatable  to  elements  of  a  task  lists.  Further, 
in  keeping  with  the  lumped  parameter  process,  the  preponderance  of  V/L  estimates  do 
not  provide  a  Level  2  damage  vector  (list  of  working/nonworking  components).  The  Level 
2  metrics  are  not  mapped  to  a  Level  3  capability  to  be  related  to  a  task  to  be  executed  at 
Level  4.  The  output  is  the  PK,  which  is  used  as  a  Bernoulli  draw  in  a  wargame.  When 
multiple  hits  are  received  on  a  target,  a  number  of  PK  events  are  computed 
individually,  assumed  to  be  actual  probabilities,  and  assumed  to  be  independent,  and 
then  combined  by  the  Survivor  Sum  rule  (see  Endnote,  p.  36). 

Returning  to  Figure  6,  we  observe  once  again  that  interactions  change  the  material/  people 
at  Level  2.  If  a  sequence  of  events  occurs,  then  the  state  of  Level  2  must  evolve  by  steps. 
To  state  it  a  different  way,  the  aggregation  of  effects  occurs  at  Level  2.  As  Level  2  evolves 
over  time,  the  capability  at  Level  3  and  effectiveness  at  Level  4  follow.  What  happens  in 
areas  of  analysis  is  that  individual  interactions  are  mapped  through  Level  2  to  Level  3 
and  more  often  to  Level  4.  The  aggregation/integration  of  multiple  interactions  are 
incorrectly  estimated  by  combining  multiple  performance  (Level  3)  or  effectiveness  (Level 
4)  metrics.  This  is  clearly  wrong,  but  hard  to  perceive  absent  a  logical  framework. 
Compounding  this  problem  is  the  fact  that  virtually  all  wargames  treat  platforms  as  black 
boxes  with  assigned  properties.  They  are  not  represented  with  working  components  that 
can  break  or  be  fixed.  Hence  for  these  simulations,  there  are  no  dynamic  Levels  2,  3,  and 
4  and,  therefore,  no  way  to  develop  combined  effects  properly! 

Further,  because  in  the  standard  wargame  PKs  are  used  as  Bernoulli  draws,  outcomes  are 
either  0  or  1.  There  are  no  intermediate  values.  Platform  performance  characterized  by 
binary  states  therefore  provides  an  inadequate  measure  of  task  performance. 

To  review  some  key  points: 
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•  The  inability  to  define  the  "PK"  metrics  objectively/  quantitatively  as  well  as  lack  of 
objective  intermediate  damage  and  performance  metrics  contributed  greatly  to  the  Live 
Fire  Program  issues  in  the  1980s. 

•  Lumped  parameter  metrics  in  general  are  problematic  with  respect  to  both 
interpretation  and  integration  with  other  parameters! 

•  Absent  context  and  intermediate  results,  the  contribution  of  each  of  the  three 
components  (physical  state  change,  capability  change,  and  change  in  mission 
challenge)  cannot  be  apportioned  to  create  data  extensibility. 

•  PKs  are  actually  binary  measures.  As  such,  they  represent  the  average  of  many 
elements  of  an  ensemble.  Comparison  of  single  members  of  the  ensemble  with  the 
average  is  not  appropriate. 

•  With  such  characteristics,  PKs  are  fundamentally  useless  for  providing  insights  to 
task  performance. 

•  However,  when  fighting  components  are  abstracted  at  the  level  of  resolution  portrayed 
in  Figure  6,  an  extensible  construct  is  formed  which  is  capable  of  describing  and 
integrating  a  large  number  of  phenomena  (see  Figure  10) 

The  issue  of  test/model  duality  to  estimate  performance  for  comparison  with  task 
requirements  is  key  to  understanding  the  prosecution  of  the  MDMP.  We've  emphasized 
the  importance  of  estimating  a  continuum  of  performance  values  for  key  platform 
capabilities.  In  APPENDIX  B,  we  provide  the  logic  for  determining  mission  readiness. 

MEASURING  AND  ESTIMATING  COSTS 

Having  introduced  the  four  front  layers  of  MMF,  Levels  1-4,  as  well  as  the  suggestion  that 
they  relate  to  natural  language  word  classes,  it  is  opportune  to  discuss  the  issue  of  costing 
warfighting.  Clearly,  the  monetary  resourcing  of  DoD  initiatives  is  among  the  most 
important  considerations.  Any  analytic  framework  unable  to  support  cost  methodology 
would  hardly  be  useful. 

There  are  two  obvious  foci  in  MMF  that  provide  links  to  measure  and  estimate  costs. 
The  first  characterizes  the  cost  of  materiel  at  Level  2.  Such  estimates,  for  example,  would 
reflect  the  purchase  costs  for  weapons.  The  second  focus  is  found  at  Level  4,  Tasks. 
From  this  perspective,  the  goal  is  to  fix  the  cost  of  actions  or  activities.  In  fact,  this 
approach  matches  the  well-established  practice  of  Activity-Based  Costing  (ABC)  found 
throughout  financial  and  accounting  worlds.  Those  interested  methodologies  to  identify, 
measure,  and  categorize  the  various  costs  in  defense  analysis  are  directed  to  Nelson  (2006) 
and  Deitz  (2009,  p.  224). 
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A  LEGO  COLLECTION  OF  MISSION  PERFORMANCE  ELEMENTS 


In  the  prior  sections,  we  have  reviewed  all  of  the  pieces  needed  to  construct  an  MDMP 
analog.  The  seven  levels  (see  Figure  5)  provide  needed  context.  The  task  definition 
and  execution  pieces  are  shown  in  Figure  10.  With  the  Universal  and  Service  task  lists,  the 
semantics  of  Level  4  are  well  established.  At  Level  2,  the  military  hierarchical  naming 
conventions  for  people  and  materiel  have  been  well  practiced  for  a  few  thousand  years! 
The  semantics  of  Level  3  need  to  follow  from  the  appropriate  tasks  defined  in  the  official 
task  lists.  That  way,  when  a  mapping  is  made  from  the  state  of  components  to  the 
corresponding  capability,  it  will  be  immediately  obvious  whether  or  not  it  meets  or  exceeds 
the  Level  4  task  requirement.  Probably  the  elements  of  Level  1 
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Figure  10.  The  key  elements  of  task  execution.  The  four  classes  of  levels  and  the 
four  classes  of  operators  and  be  thought  of  as  logical  "lego"  pieces.  Each  element 
can  be  developed  and  tested  individually  and  then  combined  in  endless 
combinations  to  define  specific  Task  Cycles. 

have  received  the  least  standardization.  Nevertheless,  the  V/L  community  has  much 
experience  in  constructing  a  range  of  ballistic  operators  to  cover  a  large  collection  of 
warhead/target  interactions.  And  there  are  many  experts  in  a  diverse  set  of  disciplines 
capable  of  quantifying  their  respective  phenomenologies. 
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THE  TEST/MODEL  DUALITY  OF  MMF 


Previously,  the  MDMP  has  been  described  as  the  accepted  warfighting  paradigm  in  which 
sequences  of  tasks  at  connected  levels  of  war  are  executing  by  assigned  entities  (people/ 
platforms).  Supporting  the  MDMP,  the  MMF  structures  not  only  how  task  sequences  are 
formulated,  but  what  entities  are  assigned  to  the  execution  of  those  tasks.  The  paradigm 
necessarily  recognizes  that  physical  entities  change  over  time  due  to  task  execution  from 
both  internally  generated  and  external  factors  as  well. 

We  posit  that,  to  be  of  use,  the  MMF  structure  must  be  adequate  not  only  to  the  theoretical 
(modeling,  calculating)  side  of  enquiry  but  to  testing  (observation,  measuring)  as  well.  In 
fact  for  model  (or  even  test)  validation  to  occur,  the  theoretical  and  test  abstractions  must 
be  fully  shared.  This  property,  the  sharing  of  a  common  abstraction  between  the  test  and 
model  worlds  has  not  been  widely  practiced  in  much  of  the  test  and  analytic  worlds.  The 
sharing  of  the  MMF  structure  should  be  one  approach  to  minimizing  that  problem. 

APPENDIX  C  discusses  the  parity  issue  in  further  detail. 

CONSTRUCTING  A  DEVELOPMENTAL  TEST 

Now  with  our  various  lego  pieces  in  hand,  we  can  construct  a  variety  of  Task  Cycles 
with  breadth  and  depth  tailored  to  questions  at  hand.  They  can  be  responsive  to  tasks 
defined  at  Level  4,  with  a  frequency  defined  by  the  Om  Operator,  and  linked  to  a  large 
class  of  chosen  effects  listed  on  the  far  right  of  Figure  10.  The  class  of  effect  calls  the 
appropriate  Oi,2  Operator,  which  causes  changes  to  occur  at  Level  2.  Next,  the  concomitant 
capability(ies)  change  at  Level  3  via  the  02,3  Operator,  and  finally  a  comparison  is  made 
with  Level  4  to  gauge  the  required  capability  for  the  next  cycle. 

A  single  Task  Cycle  (as  in  Figure  6)  might  represent  one  exercise  of  a  Developmental  Test 
(DT).  A  DT  planning  process  might  consist  of  reviewing  each  of  the  four  levels  and  each 
of  the  four  operators  to  establish  the  degree  to  which  each  is  known  for  the  context 
variables  likely  to  be  encountered  during  testing.  A  system  under  development  is 
characterized  by  many  possible  task  responses.  The  Task  Cycle  (Figure  6)  can  be  used  to 
first  list  the  set  of  system  responses  appropriate  for  performance  and  testing  purposes. 
The  particular  elements  of  the  Task  Cycle  can  be  reviewed  with  respect  to  responses  both 
from  an  M&S  as  well  as  a  testing  perspective. 


CONSTRUCTING  AN  OPERATIONAL  TEST 

As  we've  seen,  mission  threads  are  composed  of  a  sequence  of  Task  Cycles  as 
illustrated  in  Figure  3.  The  task  sequence  (or  thread),  illustrated  in  Figure  11,  can  be  used 
to  emulate  an  Operational  Test  (OT).  The  disciplines  listed  around  the  thread  point  to 
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the  variety  of  investigators  who  can  be  informed  concerning  their  particular  area  of 
expertise  and  responsibility. 
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Figure  11.  Illustrating  a  mission  thread.  A  sequence  of  tasks,  each  with  its 
attendant  Task  Cycle.  The  disciplines  surrounding  the  thread  points  to  the 
many  areas  of  DoD  research,  engineering,  costing,  and  analysis  that  can  be 
informed  by  such  an  integrated  process. 

Some  classes  of  levels/operators  might  be  of  interest  only  narrowly.  But  that  is  likely 
to  be  the  exception;  much  in  this  construct  is  shared.  All  can  use  the  characterization  of 
mission  effectiveness  based  on  a  shared  mission  build.  In  fact,  everyone  should  be  able 
to  share  in  the  levels  and  operators  from  Level  2  around  to  Level  4.  One  would  expect 
diversity  in  the  behavior  of  the  Ovi  Operator,  which  establishes  what  modeling  circles 
call  the  Time-Ordered  Event  List  (TOEL);  and  also  in  the  Level  1  interactions  and  the 
Level  1  and  Oi,2  Operator.  But  if  the  community  were  to  work  in  this  uniform  paradigm, 
everyone  could  share  the  same  effectiveness  objectives  and  have  the  means  as  well  of 
interleaving  platform  interactions,  making  possible  truly  integrated  performance 
estimates! 

A  final  element  in  establishing  standards,  commonality,  and  exclusivity  across  the  DoD 
community  is  that  this  structure  can  greatly  aid  the  process.  In  recent  years,  there  have 
been  many  bottom-up  exercises  to  establish  common  metrics  and  naming  conventions. 
But  bottom-up  processes  are  ambiguous  both  in  establishing  false  shared  metrics  (because 
the  names  might  be  the  same)  and  in  failing  to  establish  shared  metrics  (because  the  names 
aren't  the  same).  Because  the  MDMP  is  top-down,  there  is  an  inherent  traceability  down 
through  the  layers.  Having  established  the  top-down  linkages,  the  same  paths  can  be 
traced  back  up  the  logical  threads  until  different  supporting  disciplines  can  link  up  to 
common  purpose.  Thus,  all  communities  of  interest  can  focus  on  the  specific  elements 
with  clarity;  define  sharing  or  exclusivity  with  others;  and  resolve  precedence, 
dependencies,  etc. 
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To  illustrate  this  concept  another  way,  we  can  pose  this  question.  When  each  of  the 
disciplines  shown  surrounding  the  thread  looks  at  each  of  the  elements  of  the  Task  Cycles 
and  supporting  data,  to  what  extent  are  these  elements  mostly  shared,  slightly  shared  or 
mutually  exclusive?  Put  more  figuratively,  are  the  Venn  data  sets  more  like 


SYSTEMS-QF-SYSTEMS  VIA  COLLECTIVE  TASKS 

When  a  number  of  platforms  operate  together,  abstractly  they  might  appear  as  shown 
in  Figure  12.  Here  effectiveness  is  no  longer  simply  about  individual  platforms,  but  how 
a  collection  of  platforms  performs  as  a  team.  For  a  decade  or  more,  such  a  collection  of 
platforms  has  been  referred  to  as  a  System-of-Sy stems  (SoS).  SoS  are  frequently  discussed, 
particularly  with  the  view  that  teams  of  platforms  will  surely  do  better  than  individual 
platforms.  This  view  seems  plausible,  but  the  efficacy  of  and  justification  for  SoS  often 
seems  couched  in  engineering  analyses. 


Figure  12.  A  mission  composed  of  multiple  platforms  each  with  its  own 
sequence  of  Task  Cycles.  Collections  of  platforms  working  together  are 
often  referred  to  as  System-of-Systems  (SoS).  The  symbolism  to  the  right  is 
meant  to  represent  the  construct  of  a  collective  task. 

Engineering  practice  focuses  on  optional  ways  platforms  can  cooperate  as  well  as  to  show 
what  methods  for  interoperability  can  be  employed.  We  suggest  that  such  insights  are 
necessary,  but  hardly  sufficient,  to  establish  insights  into  SoS  efficacy.  In  particular  we 
note  that  in  the  MDMP  decomposition  process,  collective  tasks  are  established  by  the 
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mission  operator  prior  to  assigning  individual  tasks  to  particular  platforms.  To  use  a 
football  analogy,  individual  players  may  qualify  for  specific  positions  based  on  their 
strength,  speed,  and  athleticism.  But  in  the  end,  team  success  is  based  on  the  execution 
of  actions  defined  by  the  team  playbook.  Without  regard  to  team  activity  based  on  the 
plavbook,  the  value  of  the  team  is  in  question.  Similarly,  without  the  establishment  of 
collective  tasks  as  an  integral  part  of  today's  warfighting  activity,  the  effectiveness  of  the 
DOTMLPF  parameter  space,  including  individual  people  and  platform  capability,  is  also 
in  doubt. 

LINKING  IT  ALL  TOGETHER 

Previously  we  discussed  the  notion  that  vertical  linkage  between  organizational  echelons 
and  levels  of  war  is  supported  by  the  practice  of  capturing  the  externally  directed  Level  7, 
Mission,  and  the  restated  mission  derived  from  analysis  of  the  directed  mission.  This 
practice  reflects  the  doctrinal  idea  of  "Nested  Concepts".  The  Operations  Process,  ADRP 
5-0,  describes  nested  concepts  as  .  .  .  "a  planning  technique  to  achieve  unity  of  purpose 
whereby  each  succeeding  echelon's  concept  of  operations  is  aligned  by  purpose  with  the 
higher  echelon's  concept  of  operations.  An  effective  concept  of  operations  describes  how 
the  forces  will  support  the  mission  of  the  higher  headquarters  and  how  the  actions  of 
subordinate  units  fit  together  to  accomplish  the  mission.  Commanders  do  this  by 
organizing  their  forces  by  purpose.  Commanders  ensure  the  primary  tasks  for  each 
subordinate  unit  include  a  purpose  that  links  the  completion  of  that  task  to  achievement 
of  another  task,  an  objective,  or  an  end  state  condition"  (Department  of  the  Army,  2012). 

The  dotted-blue  arrows  in  Figure  13  illustrate  the  top-down  decomposition  through  a 
Level  4  primary  task  at  one  echelon  linking  to  the  Level  7  directed  mission  for  a 
subordinate  unit  at  a  lower  echelon.  Likewise,  the  solid  red  arrows  illustrate  the  bottom- 
up  linkage  between  mission  effectiveness  of  a  subordinate  unit  and  the  mission  of  the 
higher  headquarters  that  relies  on  the  success  of  it  subordinates  for  its  own  overall 
mission  effectiveness. 

IS  THIS  SIMPLY  THEORY? 

The  MMF  has  proven  to  be  both  practical  and  applicable  to  a  range  of  problem  sets.  Four 
projects  are  cited  briefly: 

•  Testing  in  a  Joint  Environment  (TJE):  The  objective  of  the  project  was  to  generate  a 
rational  plan  for  operational  test-range  employment  and  investment  for  a  complex  SoS. 
The  MMF  served  three  purposes:  1]  to  organize  available  information  pertinent  to  OSD 
T&E,  2]  to  analyze  that  information  to  identify  T&E  capability  gaps  in  a  Joint 
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Environment,  and  3]  to  provide  inputs  for  a  Rough  Order-of-Magnitude  (ROM) 
estimation  for  the  corrective  investment  (Payne  et  al.,  2005).  The  MMF  also  informed  a 
detailed  analysis  of  a  mission  decomposition  and  functional  capabilities  crosswalk  to  set 
the  OT  context. 


NATIONAL 


Platform  History 


Figure  13.  The  MMF  connectivity  by  level-of-war  (vertically)  and  in  time 
(horizontally).  The  MMF  has  been  used  to  provide  an  abstract  formalism  for 
the  MDMP.  Its  iconic  representation,  shown  on  the  right,  is  used  recursively  to 
provide  logical  structure  for  a  top-down  (dotted  blue)  decomposition  process 
and  a  bottom-up  (solid  red)  assessment  process. 

•  Stability  and  Reconstruction  Operations  (SRO)  Micro-Experiment:  This  micro¬ 
experiment  was  designed  to  identify  capability  gaps  associated  with  an  FCS 
equipped  brigade  combat  team  (BCT)  performing  an  SRO  mission.  The  MMF  study 
team  developed  an  analytical  model  using  the  EXTEND  discrete  event  simulation 
environment.  The  model  enabled  the  team  to  simulate  multiple  execution  runs  of  a 
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180  day  SRO  mission  window  and  identify  tasks  that  could  not  be  completed  to 
standard  due  to  the  loss  or  degradation  of  assigned  systems  (UAMBL,  2005). 

•  Managing  Intelligence  Resources:  This  work  addressed  the  challenge  of  how  to  deploy 
and  utilize  limited  intelligence,  surveillance  and  reconnaissance  (ISR)  resources  most 
effectively  in  joint-forces  operations  (Gomez  et  ah,  2007).  Using  the  MMF,  modern 
military  doctrine  was  captured  in  a  semantically  formal  representation,  allowing  sensors  and 
other  ISR  resources  to  be  assigned  to  a  mission  through  matchmaking  reasoning. 

•  Demonstration  of  Mission-Based  Operational  Assessment:  A  high-resolution 

computer  simulation  was  developed  to  explore  the  effectiveness  of  a  networked, 
company-level  fighting  unit.  In  the  context  of  collective  and  platform  mission  tasks,  the 
analysis  tracked  1]  the  company's  ability  to  continue  its  mission  as  a  networked  SoS 
while  suffering  loss  of  capability  in  selected  components,  and  2]  the  impact  of 
degraded  system  functions  on  critical  mission  tasks.  This  demonstrated  that  mission 
accomplishment  at  higher  levels  of  combat  can  be  seamlessly  linked  to  functional 
state  changes  in  low-level  components  (Ward  et  ah,  2012). 

SUMMARY 

We  have  described  a  framework  that  builds  on  the  well-established  MDMP.  This 
methodology  is  capable  of  spanning  vertically  all  levels  of  war  and  horizontally  over  the 
complete  time  span  of  a  mission.  The  MDMP  has  been  enriched  through  the  use  of  the 
MMF,  which  has  extended  the  semantic  formalism  to  key  additional  elements  which  exist 
across  all  levels  of  war  as  illustrated  in  Figure  13. 

Full  operational  context  is  established  in  support  of  sequential  task  cycling  for  all 
materiel/people  players  and  all  supporting  disciplines.  When  the  "lego"  elements, 
described  in  Figure  10,  are  developed  at  this  level  of  resolution,  they  can  be  combined  in 
many  ways,  with  great  extensibility.  This  makes  possible  a  single  logical  construct  which 
can  serve  the  warfighter/operator  as  well  as  the  modeling/simulation  activities.  This 
same  construct  can  be  used  across  many  disciplines—  theory,  modeling  and  testing,  as 
indeed  it  must  if  theoretical  abstractions  and  testing  practice  are  ever  to  mutually  reinforce 
each  other  constructively.  Only  by  such  practice  can  validation  (comparisons  between 
tests  and  models)  approach  its  potential  and,  in  the  sense  of  SoS,  enable  DoD  activities  to 
finally  become  greater  than  the  sum  of  their  parts! 

FINAL  PERSPECTIVES 

DoD  Testing/Modeling:  Some  of  the  analytic  methods,  tools,  and  techniques  used  today 
have  roots  more  than  50  years  old.  The  concepts  and  processes  employed,  in  the  main, 
have  arisen  independent  of  one  another,  mostly  in  uncoordinated,  bottom-up  activities. 
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We  therefore  have  an  array  of  analytic  areas  characterized  by  collections  of  metrics  in 
which  commonality  or  exclusion  across  disciplines  is  neither  defined  nor  generally  known. 
Because  standard  language  is  often  absent,  there  is  imperfect  communication  of  ideas  and 
even  metrics  between  and  within  communities  of  practice.  Specific  analytic  and  test 
activities  are  prosecuted  in  isolation  without  the  ability  to  integrate  them  holistically.  We 
add  that  this  is  not  a  limitation  imposed  by  software  "integration"  or  "engineering."  This 
is  a  consequence  of  analysts  and  coders  operating  absent  a  clear  concept  of  the  elemental 
logic  pieces  of  warfighting  and/or  materiel  capability  and  how  the  many  pieces  should 
properly  fit  together. 

Global  Abstract  Structure:  There  are  some  ironies  here.  Lanchester  established  his  famous 
LER  methods  for  estimating  warfighting  outcome  nearly  a  century  ago.  Contemporaneous 
with  the  initiation  of  World  War  II,  methods  in  operations  research  and  system  analysis 
exhibited  explosive  growth!  But  the  way  wargames  are  prosecuted  today  remain  essentially 
identical  to  the  concepts  and  methods  established  circa  1960!  If  you  enquire  as  to  what 
overall  logic  or  structure  informs  the  DoD  either  globally  at  the  higher  levels  of 
warfighting,  or  narrowly  at  lower  levels  where  people/materiel  more  typically  collide 
with  task  execution,  it  would  seem  we  have  none!  How  could  this  be?  Maybe  in  our 
collective  rush  to  get  the  "what"  of  so  many  important  activities  accomplished,  we  may 
have  skipped  past  the  "why"  and  "how"  of  what  we  are  doing! 

The  absence  of  what  might  be  called  explicit  analytic  structure  has  ramifications.  As 
noted  previously,  the  V/L  models  used  from  the  late  1950s  through  today  are  substantially 
based  on  lumped-parameter  averages,  where  damage,  capability,  and  utility  are  conflated. 
One  set  of  metrics  used  in  direct-fire  ballistic  V/L  analysis  goes  under  the  label  of 
"Expected  Loss-of-Function."  One  might  think  that  "Function"  has  to  do  with  capability 
or  performance,  but  it  actually  refers  to  an  unclear  notion  of  mission  utility,  which  mashes 
physical  damage,  performance  decrement,  and  mission  utility  (averaged  over  a  range  of 
unspecified  missions)  into  a  number  in  the  range  of  zero  and  one!  And  the  word 
"Expected"  would  seem  to  apply  to  an  ensemble  average  of  some  sort,  but  there  is  no 
basis  for  that  descriptor  either!  This  imprecision  in  metric  specification  contributed  to  the 
difficulties  encountered  by  the  Army  in  its  Bradley  Live  Fire  program.  Also  binary  kill 
values  (i.e.,  zero  or  one)  yielded  by  the  lumped  parameter  approaches  are  unsuitable  to 
gauge  task  accomplishment  either  at  the  single-  platform  level  or  when  combined  through 
appropriate  logic  (see  Appendix  B),  at  the  SoS  level.  Thus,  to  the  extent  wargames  or 
other  simulations  fail  to  represent  the  state  of  platform  component-level  health  (Level  2 
state  space),  there  is  no  way  to  examine  the  ways  in  which  systems  degrade  or  reset! 

But  on  the  positive  side,  the  challenges  of  ballistic  live-fire  programs  three  decadesago 
required  the  Army  analytic  community  to  develop  the  modeling  granularity  portrayed  in 
Figure  6.  As  we  have  come  to  understand  more  completely  this  abstraction,  it  appears 
increasingly  to  be  a  requisite  member  of  an  interdisciplinary  framework. 
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Absence  of  Effectiveness  Definitions/Connections:  Effectiveness  analyses  (e.g., 

requirements,  wargames,  test,  evaluation  activities)  are  not  structured  in  a  way  that  clearly 
relates  system  requirements  to  operational  necessity  using  approved  doctrinal  terms. 
Acquisition  activities  typically  proceed  without  standard,  shareable  performance  and 
effectiveness  metrics.  Weapons  programs  are  typically  pursued  without  detailed 
explanation  of  the  value  added  and  associated  risk  assessments  in  operational  context. 
This  shortcoming  on  the  acquisition  side  of  the  community  is  ironic  because  the 
warfighters  through  the  MDMP  have  long  structured  their  mission-  related  activities. 
And,  since  the  mid-90s,  the  development  of  standard  task  lists  has  enabled  semantic 
clarity!  But  the  acquisition  community  still  often  ignores  the  task  construct,  preferring  to 
grapple  with  capabilities. 

Let's  consider  again  SoS.  We  are  unaware  of  any  SoS  analysis  based  upon  actual 
traceability  back  to  task-based  requirements  in  two-sided,  multiplatform  combat. 
Tests/analyses  all  begin  with  a  "Requirements  Statement"  (or  similar  document)  in 
which  SMEs  opined  the  key  metrics,  but  without  the  benefit  of  logical  linkage  of 
interconnected  team  performance  (i.e.,  SoS)  actually  tied  to  mission  threads  evolving  over 
time.  The  time  dimension  is  critical  not  only  to  specifying  what  must  be  done  in  the 
mission,  but  also  characterizing  the  ever-changing  capabilities  of  the  SoS  team  based  on 
considerations  of  accrued  damage,  repair,  logistics,  physics  of  failure,  resupply,  and  many 
other  possible  factors.  The  key  to  providing  this  logical  linkage  of  interconnected  team 
performance  lies  in  the  use  of  collective  tasks  to  describe  and  understand  the  linkage. 
As  we've  emphasized,  collective  tasks  rely  on  the  coordinated  and  integrated  performance 
of  a  team  of  systems  assigned  to  the  subordinate  and  supporting  tasks  that  together  form 
the  collective  task. 

Consider  a  football  metaphor  to  illustrate  the  point.  One  can  think  of  the  individual 
players  as  systems  within  the  larger  SoS  that  is  the  team.  Each  player  has  a  unique  set  of 
attributes  (e.g.,  speed,  size,  strength,  passing  skill)  that  must  be  coordinated  and  integrated 
in  time  and  space  for  every  play  of  the  game.  Called  plays  can  be  compared  to  collective 
tasks  with  each  player  responsible  for  executing  one  or  more  individual  assignments/tasks 
(e.g.,  blocking,  passing,  receiving).  Assessing  individual  football  players  on  their  ability 
to  perform  their  individual  assignments  is  not  a  valid  predictor  of  team  success.  It  is 
only  when  players  assemble  on  the  field  and  execute  plays  as  a  team  that  assessments 
of  potential  team  success  can  be  made.  The  effects  of  accrued  injuries,  fatigue,  medical 
treatment,  rest,  and  recuperation  are  well  understood  in  terms  of  potential  impact  on  the 
effectiveness  of  our  favorite  football  teams. 

To  the  best  of  our  knowledge,  the  collective-task  construct  stands  alone  as  the  warfighter's 
single  defining  representation  of  "combat  team"  effectiveness!  Absent  this  constructive 
guiding  linkage,  there  are  no  activities  within  the  technical  design  and  engineering 
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communities,  no  matter  the  skill  and  dedication  expended,  that  can  deliver  the  effective, 
suitable,  and  survivable  materiel  our  warfighters  deserve. 

Remedies:  Improvements  must  proceed  from  two  directions.  By  applying  the  MMF 
to  initial  analysis  of  capstone,  operating,  and  functional  concepts,  the  requirements 
community,  grounded  in  operational  context,  can  generate  products  (i.e.,  mission  threads, 
mission  task  lists)  that  are  understandable  to  and  immediately  usable  by  the 
technical/engineering  community.  Further  application  of  the  MMF  is  needed  to  analyze 
and  generate  products  for  a  set  of  reference  missions  that  provide  that  context  and  give 
full  consideration  to  the  whole  DOTMLPF  parameter  space,  not  just  the  materiel  piece. 
The  technical/engineering  community  must  learn  how  to  link  their  parameter  spaces  back 
to  the  warfighter  mission  threads.  It  is  shown  in  Deitz  et  al.  (2009,  page  120,  ff.)  that 
the  mission  threads  are  necessary  to  evaluate  the  survivability  of  networked  SoS;  and  the 
described  method  relates  to  any  attribute,  not  just  survivability.  It  is  also  shown  that 
system  or  SoS  technical  performance  characteristics  (capabilities,  component  fault  trees) 
can  be  related  directly  to  mission  requirements.  Most  of  the  studies  and  analyses 
supported  by  MMF  to  date  have  been  funded  and  executed  to  focus  on  specific  questions 
at  hand.  As  a  result,  the  tools  used  to  apply  aspects  of  the  MMF  have  typically  been 
developed  in  a  " quick  and  dirty"  fashion  to  suit  relatively  narrow  purpose.  Serious 
consideration  needs  to  be  given  to  applying  the  lessons  learned  to  date  in  order  to 
develop/integrate  an  MMF  tool  set  to  help  automate  the  analytical  processes  and  generate 
products  in  standardized,  machine  executable,  as  well  as  human  readable  formats  that 
can  be  readily  shared  and  leveraged  within  and  between  the  two  communities. 

APPENDIX  A:  MAPPINGS 

Mathematical  levels  or  spaces  embody  a  useful  concept  to  aid  in  the  understanding  a 
process  as  complicated  as  the  MDMP.  A  simple  way  to  begin  is  to  examine  Figure  Al. 

In  this  physical  example,  three  individual  shadows  of  an  opaque  object  are  shown  on  three 
different  planes.  Each  shadow  is  a  valid  "projection"  of  the  3-D  object,  but  also  each 
shadow  is  an  incomplete  representation  of  the  object.  This  is  an  example  of  a  projection 
from  a  higher  (more  complete)  space  to  a  lower  (less  complete)  space.  In  general, 
mappings  can  occur  only  from  higher  to  lower  spaces;  thus,  they  are  not  invertible.  It  is 
this  property  of  noninvertibility  that  makes  top-down  processes  so  critical  to  providing  the 
most  complete  array  of  parameter  options  when  instantiating  a  multidimensional 
framework.  By  contrast,  bottom-up  processes  are  appropriate  for  the  (time)  execution 
of  mission  threads,  but  not  for  establishing  in  isolation  a  structure  (with  metrics)  for  a 
parameter  space.  When  such  ad  hoc  methods  are  followed,  there  is  no  single  reference 
object  for  varied  perspectives  to  seek  common  structure.  Conceptually  speaking,  it  is  this 
property  that  can  make  it  difficult  for  workers  in  different  disciplines  to  understand  the 
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extent  to  which  processes  and  metrics  familiar  to  them  are  shared  (and  how)  with  other 
groups. 


Figure  Al.  Projections  in  spaces  of  differing  dimensions.  An  opaque  physical 
object  is  located  in  three-dimensional  space.  If  a  light  is  projected  from  right  to 
left,  parallel  to  the  Y  axis,  a  shadow  is  cast  on  the  X-Z  plane.  Likewise,  if  a  light 
is  projected  directly  downward,  parallel  to  the  Z  axis,  a  shadow  will  be  cast 
on  the  X-Y  plane.  So  also  a  light  beam  projected  along  the  X  axis  casts  another 
shadow  on  the  Y-Z  plane.  In  this  case,  the  three  projections  are  different,  but 
each  provides  valid,  but  incomplete,  information  concerning  the  3-D  object. 


Figure  A2  builds  on  the  example  of  Figure  Al,  and  casts  it  into  an  abstract  space.  The  axes 
do  not  represent  a  simple  Cartesian  coordinate  system.  In  this  example,  some  engineers 
are  contemplating  the  design  of  a  truck.  By  virtue  of  the  material  properties  of  the  truck, 
it  has  certain  capabilities  to  support  logistical  tasks,  to  move  and  maneuver  with  particular 
mobility  characteristics,  and  to  survive  or  not  when  struck  by  a  ballistic  threat.  The 
philosophic  issue  here  is  which  of  the  properties,  fully  defined  in  the  actual  (3-D)  vehicle, 
have  shared  projections  to  the  three  " planes."  In  other  words,  what  basic  truck  properties 
map  to  each  of  the  three  areas,  how  are  they  shared,  and  how  can  they  be  manipulated  to 
an  optimum  design? 
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Mobility  Plane 


Figure  A2.  An  example  of  a  projection  from  3-D  to  2-D  space.  An  abstract 
example  of  projections  from  a  higher  space  to  three  lower  spaces.  This  issue  is 
what  truck  properties  (from  the  real  3-D)  space  are  identically  projected  to  the 
three  subspaces.  Knowledge  of  the  sharing  or  exclusivity  of  this  can  impact 
performance  and,  ultimately,  platform  effectiveness. 

But  there  is  an  important  caveat  as  we  attempt  to  illustrate  the  MDMP  as  an  even  more 
abstract  set  of  spaces  in  Figure  A3.  Because  the  MDMP  is  built  decompositionally,  there 
do  not  exist  complete  spaces  of  information  at  one  level  of  war  simply  to  be  mapped  to  a 
lower  space.  Many  connections  are  inferential  and  an  important  part  of  the  war-  planner 
conceptual  activity.  So,  in  the  mission  build  process,  whether  the  linkages  are  explicitly 
conveyed  or  implicitly  inferred,  this  methodology  provides  significant  advantages  in 
understanding  the  relationships  among  the  complex  parameter  space. 

Our  challenge  across  the  numerous  "Defense  Analytic  Challenges"  is  to  understand  how 
the  many  elements  fit  together,  and  where  commonality  across  our  many  disciplines  needs 
to  be  identified  not  only  for  matters  of  efficiency  but  so  we  can  achieve  optimum 
design  and  process  control  across  the  many  activities  for  which  the  logical  linkages  are  not 
explicit. 
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Figure  A3.  Building  on  the  abstraction  of  Figure  A2.  Defense  Analytic 
Challenges  represent  a  higher  space  to  the  various  disciplines  displayed  as 
populating  lower-level  spaces. 


APPENDIX  B:  DETERMINING  MISSION  READINESS 

Applying  the  MMF  to  operational  analysis  is  akin  to  a  math  teacher  requiring  the  student 
to  "show  his  work,"  including  all  of  the  intermediate  steps,  rules,  and  axioms  applied  to 
reach  the  final  answer.  For  the  demonstration  of  mission-based  operational  assessment 
(Ward  et  al.,  2012),  the  study  team  needed  to  describe  clearly  the  steps  in  the  decision¬ 
making  process  used  to  determine  collective  task  and  mission  impact  resulting  from 
changes  in  the  ability  of  platforms  (aka  systems)  to  execute  assigned  tasks  to  standard. 
Doing  so  was  critical  to  developing  the  software  logic  needed  to  update  the  status  of 
platform  tasks,  collective  tasks,  and  the  overall  mission  during  simulation  run  time.  The 
box  in  the  upper-right  corner  of  Figure  B1  illustrates  the  process  flow  for  the  collective  task 
titled  "Manage  tactical  information".  This  process  flow  is  an  example  of  the  theoretical 
description  in  Figure  12  put  to  practice.  Because  most  missions  are  composed  of  multiple 
sets  of  interdependent  collective  tasks,  it  is  necessary  to  describe  a  process  flow  to  link  the 
changing  status  of  critical  collective  tasks  to  the  resulting  impact  on  the  overall  mission. 
The  main  body  of  Figure  B1  below  illustrates  this  linkage. 
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Figure  Bl.  Dependency  of  collective  tasks  on  platform  tasks.  Linkage  from 
Platform  Task  Capability  to  Collective  Task  Capability  to  Mission  Impact 

The  process  relies  on  data  collected,  stored,  and  organized  in  accordance  with  the  MMF 
structure,  based  on  the  results  of  detailed  top-down  analysis  of  the  mission.  This  facilitates 
the  ability  to  " drill  down"  and  determine  the  specific  factors  causing  task  or  mission  failure 
and/or  increasing  the  risk  of  failure. 

APPENDIX  C:  TEST/ABSTRACTION  PARITY 

Decision  making  in  the  DoD  should  be  based  on  a  process  of  Knowledge  Formation.  Our 
senior  managers  should  be  provided  appropriate  information  to  sift,  filter,  analyze,  and 
evaluate  various  options.  This  activity  is  represented  at  the  top  of  Figure  Cl.  The  key 
point  here  is  that  there  are  two  paths  to  Knowledge  Formation.  One  is  to  observe;  the 
other  to  theorize.  In  addition  to  observation,  the  former  path  uses  exercises,  measures,  and 
tests.  On  the  right-hand  side,  the  approach  is  to  calculate,  model,  represent,  and  simulate. 

Evaluation  ("E")  takes  place  in  the  Knowledge  Formation  process.  Testing  ("T")  resides 
on  the  left-hand.  Observe  side  of  the  diagram.  Modeling  &  Simulation  (M&S)  live  on  the 
right-hand  side. 
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Figure  Cl.  A  vision  for  DoD  decision-making.  Key  knowledge  is  based  on  two 
complementary  paths,  observation  and  theory.  Requirements  flow  top-  down 
(per  blue  arrows);  Information  flows  bottom-up  (per  red  arrows).  Without  an 
Abstraction,  or  a  way  of  "thinking  about  an  activity",  a  tester  has  no  idea  what 
to  measure,  what  instrumentation  to  utilize,  or  how  to  process  the  results. 

Most  would  expect  Abstraction  to  be  a  central  part  of  the  right-hand  Calculate  process. 
After  all,  simulations  are  based  on  abstractions  implemented  in  code  and  executed  by 
computer.  But  Abstraction  is  just  as  much  a  critical  part  of  Observation  and  Testing  as 
well.  For  without  an  abstraction,  or  a  way  of  "thinking  about  an  activity,"  a  tester  would 
have  no  idea  what  to  measure,  what  instrumentation  to  use,  or  how  to  process  the  results. 

What's  important  here  is  that  1]  both  sides  have  abstractions  and  2]  that  they  are 
harmonized!  If  not,  an  activity  on  one  side  is  logically  incompatible  with  its  opposite  in 
kind  of  metric,  level  of  granularity,  time  resolution,  or  some  other  key  property.  And 
without  the  ability  to  compare  "apples-to-apples,"  validation  cannot  take  place.  Without 
validation,  whatever  actual  number  of  tests  are  performed,  it  is  unlikely  that  they  can  be 
generalized  to  new  and  different  contexts  where  the  next  mission-of-interest  may  take 
place.  Similarly  from  the  other  side,  absent  validation,  the  credibility  of  theoretic  and 
computer  exercises  remains  uncertain  and  of  minimal  value. 
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The  need  for  a  single,  unified  abstraction,  appropriate  for  both  paths  to  knowledge,  is  the 
underlying  philosophy  upon  which  the  MMF  is  based. 

ENDNOTE 

The  Survivor  Sum  Rule  is  based  on  two  key  assumptions:  first,  the  parameters  being 
used  are  true  probabilities.  Second,  they  are  independent.  The  computation  is 
straightforward : 


Pk  Total  =  1  —  {  [  1  —  PK1  ]  X  [  1  —  Pk2  ]  X  [  1  —  PKn  ]  } 

In  practice,  particularly  in  the  vulnerability  community,  few  metrics  that  are  claimed  as 
probabilities  are  actually  so!  And  as  to  independence,  the  likelihood  of  multiple 
parameters  exhibiting  this  property  is  low  as  well.  Whether  engaged  in  a  complex  mission 
where  many  activities  are  occurring,  or  evaluating  complex  war  machines  with  many 
moving  parts,  there  is  a  high  likelihood  that  the  constituent  pieces  are  closely  linked  and 
highly  interdependent.  It  is  this  cooperation  that  underwrites  the  performance  and 
success  of  military  activities  and  makes  dubious  any  argument  that  any  of  their 
characterizations  are  independent,  one  from  another! 
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